Natural Language Interface Put in Perspective: Interaction of Search Method and Task Complexity

A 2x2 mixed design experiment (N=52) was conducted to examine the effects of search method and task complexity on users’ information-seeking performance and affective experience in an e-commerce context. The former factor had two within-participants conditions: keyword (KW) vs. natural language (NL) search; the latter factor had two between-participants conditions: simple vs. complex tasks. The results show that participants in the complex task condition were more successful when they used KW search than NL search. They thought the tasks were less difficult and reported more enjoyment and confidence with KW search. In the meantime, simple task participants performed better when they used NL rather than KW search. They also perceived the tasks as easier and more enjoyable, and had higher levels of confidence with the results, when NL was used. The findings suggest that NL search is not the panacea for all information retrieval tasks, depending on the complexity of task. Implications for interface design and directions for future research are discussed.